Device and method for seat supervision in motor vehicles

ABSTRACT

A device for seat supervision in motor vehicles has a pulsed monochromatic light source for the short-time illumination of a supervised space. An image sensor array (ISA) is also provided for recording an image of the supervised space. The ISA is provided with a global electronic shutter unit which permits the illumination time of the ISA to be adjusted independently of a readout clock of the ISA. A synchronization unit for synchronizing the operation of the pulsed laser source and the ISA is provided. Also provided is a control unit for controlling the light source and the ISA so as to create a first image of the supervised space during illumination of the same at a first time and to create a second image of the supervised space without illumination of the same at a second time. Finally, a unit for creating a difference image from the first and second images is also provided.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a device and a method for seatsupervision in motor vehicles and in particular to a device vice and amethod for optical seat supervision which are suitable able forcontrolling the activation state of one or more airbags.

2. Description of Prior Art

A centrally situated airbag should fire in the event of an accident ifthe seat is occupied by a person in the upright position. If a personbends forward too close to the airbag or if there is a child's seat orsome other object on the seat, so that a person or a child could beinjured or the airbag be triggered unnecessarily, this situation must bedetected by a seat supervision system so as to prevent the airbag fromfiring or to modify the unfolding dynamics of the airbag.

The advantages of an airbag far outweigh potential disadvantages.However, an airbag can only provide protection with full efficacy if theoccupant of the motor vehicle is belted up, so the airbag must beregarded as a supplementary unit of the so-called restrain system, SRS(SRS =Supplemental Restrain System). The airbag supports the restrainingaction of the belt in the event of a frontal collision and preventsdirect impact of the thorax, neck and head with the dashboard.

An airbag normally fires 10 to 40 ms after the collision has occurred ifthe change in speed resulting from the collision exceeds a specifiedvalue, 25 km/h in Europe and 15 km/h in the USA. The airbag envelopethen emerges from the airbag cover at a speed of 250 to 300 km/h. Thetime taken to inflate the airbag is 25 to 40 ms, which is faster thanthe eyelid reaction time. Only after this inflation time does the airbagprovide its full protection. If, at the moment an accident occurs, anoccupant is not in the normal sitting position, a situation which isreferred to as OOP (Out of Position), or if there is a child's seat onthe front passenger seat, the high acceleration rates during theunfolding phase of the airbag, which can be of the order of 100 G, canresult in serious injuries, independently of whether the change of speedwhen the airbag is fired is low or high. Thus it is possible that theconsequences of an accident which would not be that serious may be madeworse. The airbag can only exhibit its full efficacy when the occupantis situated at a sufficient distance from the airbag cover. In thenormal sitting position the distance between the driver's chest and thehub of the steering wheel is usually about 30 cm, whereas the distancebetween the front passenger's chest and the airbag cover is usuallyabout 50 cm, the normal sitting positions varying according to theinterior configuration of the vehicle.

In the field of technology dealt with here a plurality of methods forseat supervision in motor vehicles is already known by means of whichairbag activation state control can be effected on the basis of seatsupervision.

First of all, it is possible to deactivate and activate the airbagmanually so as to avoid the chance of being exposed to danger by theairbag in certain special cases. However, with this approach it is easyto forget the need to activate the airbag. Using a pressure sensorarrangement to measure the weight exerted on a seat of the motor vehicleis a method which is often used to detect seat occupancy, theactivation/deactivation of the airbag being effected on the basis of themeasured weight. With this method, however, children's seats which weighless than the weight threshold employed are not detected. Furthermore itis possible for the airbag to be activated unnecessarily if there is aload on the front passenger seat.

It is also possible to detect seat occupancy by performing a capacitivemeasurement of the object, usually a dielectric, which is situatedbetween the seat and the instrument panel. For this purpose electrodesare located in the seat and in the instrument panel. With the aid of thepermittivity measurement it is possible to detect the occupancy state ofa seat, though an error-free distinction between a human being and anobject will not generally be possible.

Also known is the practice of employing propagation time measurementmethods using active signal sources, e.g. radar signal sources or laserlight sources, to survey seat surfaces to obtain a “profile” for theoccupancy evaluation.

Methods for children's seat detection are also known which are based onthe use of special children's seats equipped with transpondertechnology. These transponders in the children's seats deactivate theairbag. With these methods based on transponder technology it is not,however, possible to detect either foreign children's seats or vehicleoccupants.

From the article “Future Applications of Microsystem Technologies inAutomotive Safety Systems” by P.Steiner and S.Schwehr, inRicken/Gessner: Advanced Microsystems for Automotive Applications 1998,Springer Verlag Berlin, 1998, a passenger state detection method isknown in which a person endangering zone immediately in front of anairbag is defined, this zone being supervised at various points by meansof active monochromatic light sources (LEDs) and traditional CCD imagesensors. When an occupant of the motor vehicle abandons the normalposition, which is situated at a typical distance from an airbag module,and thus possibly comes too close to the airbag, the system described inthe above article deactivates the airbag or reduces the firing power ofthe airbag. According to this publication the evaluation of theoccupancy detection follows the triangulation principle, whereby thelocation of an object can be established from the position of a lightspot in the image plane of a camera. The triangulation principle isknown in this area of technology.

Another alternative method for selectively controlling airbag activationand deactivation is to define a person endangering zone immediately infront of the airbag and to supervise this zone with the aid of a lightcurtain or a light barrier. If the light curtain or the light barrier isinterrupted, indicating that someone is in the danger zone, the airbagis deactivated.

The major problem with the above-cited optical methods for seatsupervision using active light sources arises because of the extremedifferences in illumination which can occur in a motor vehicle. Theillumination can vary between 1 mlx on a dark night and 100 klx indirect sunlight. Especially in the case of simultaneous direct sunlight,the light beams emitted into a supervised space by active monochromaticsources can no longer be resolved. Furthermore, as a consequence of thestatuary, country-specific safety regulations for guaranteeing theradiation safety of laser beams, the monochromatic light source may onlyemit a small amount of light energy.

SUMMARY OF THE INVENTION

It is the object of the present invention to provide a device and amethod for seat supervision in motor vehicles which permit rapiddetection of seat occupancy as a means of activating and deactivating anairbag under all illumination conditions.

According to a first aspect of the invention, this object is achieved bya device for seat supervision in motor vehicles which features a pulsedmonochromatic light source for the short-time illumination of a space tobe supervised. In addition an image sensor array for recording an imageof the supervised space is provided which has a global electronicshutter unit which enables the illumination time of the image sensorarray to be set independently of the readout clock of the image sensorarray. Also provided is a synchronization unit for synchronizing theoperation of the pulsed laser source and of the image sensor array. Thedevice according to the present invention also has a control unit forcontrolling the light source and the image sensor array so as to createa first image of the supervised space during illumination of the same ata first time and to create a second image of the supervised spacewithout illumination of the same at a second time. Finally, a unit isalso provided to create a difference signal from the first and thesecond image.

The device according to the present invention is suited, on the onehand, to the use of the optical light section principle so as todetermine the contour of an object on a seat under supervision and thusto identify the object on the seat. On the other hand the deviceaccording to the present invention is suited to determining the distancebetween a person or an object and an airbag, so that if a personsuddenly makes a movement towards the airbag this can be deactivated ifit is approached too closely. The preferred method here is to use amultizone investigation which permits power-adapted firing of theairbag.

The image sensor array of the device for seat supervision according tothe present invention is preferably a CMOS/CCD image sensor with globalshutter function which is affixed together with a monochromaticillumination source at a suitable location in the motor vehicle, e.g. inthe rooflining, on the instrument panel or in the cowl. Global shutterfunction means that a shutter unit is provided which permits all thesensor elements of the sensor array to be shuttered electronically atthe same time.

According to the present invention a scene is evaluated using differenceimage signals resulting from an image recorded with very short-timepulsed active illumination and an image recorded without pulsedillumination. The pulsed and the unpulsed image, i.e. the image withoutillumination by the light source, are recorded consecutively withminimal time lapse. The device according to the present invention thusoperates substantially trouble-free even under extreme ambientconditions. Preferably an interference filter located in front of theimage sensor array and which is tuned to the wavelength of the lightbeam emitted by the monochromatic illumination source can also be usedso that a considerable part of the interfering light can also besuppressed.

The resolvable intensity differences in the difference image created inthe present invention can subsequently be evaluated using conventionalanalysis methods employed in digital image processing and patternrecognition. Experience gained in the area of industrial measurementtechnology with light section systems under defined illuminationconditions is particularly useful here. Triangulation methods arepreferably employed to determine the distance of an occupant, ifidentified as such, from the airbag, the activation state of the airbagthen being controlled on the basis of this distance determination.

According to a second aspect of the present invention, the above objectis achieved by a method for seat supervision in motor vehicles wherein asupervised space is first illuminated by a pulsed monochromatic lightsource employing an illumination time in the microsecond or nanosecondrange. During this illumination a first image of the supervised space isrecorded by means of an image sensor array. A second image of thesupervised space without illumination of the same by the light source isrecorded a maximum of 10 ms before or after recording the first image. Adifference image is then created from the first and second images.

The present invention is based on the finding that trouble-free seatsupervision in motor vehicles can be achieved by using devices andmethods which create a difference image from a first image recordedduring an illumination and a second image recorded without illumination.For the creation of such a trouble-free difference image signal suitablefor signal evaluation certain conditions must be met. First, short-timelight pulses in the microsecond or nanosecond range must be used for theillumination so that the highest possible light power can be deliveredwith minimal light energy. In addition the time intervals between theimages recorded with pulsed and unpulsed light must be as short aspossible so as to minimize disturbances caused by object movement.Furthermore, globally synchronized illumination of the image sensor isessential in order to minimize movement fuzziness and the falsificationsin the difference image signal which result therefrom.

In the device according to the present invention the above conditionscan be realized through the use of an image sensor array with a globalelectronic shutter unit and a synchronization unit for synchronizing theoperation of the pulsed light source and the image sensor array.

In known systems in the field of optical supervision technology, imagerecording methods using a continuous, active, structured illuminationare employed. This makes it possible to achieve defined illuminationconditions, a precondition for robust image evaluation. This approach isfollowed in multifarious areas of industrial image processing for shapetesting and detection. In contrast, the present invention employs pulsedlight sources with very short illumination times in combination with animage sensor array which permits sufficiently short global illuminationtimes and which is synchronized with the active illumination source.This first makes it possible, even under extreme illuminationconditions, to radiate the structured light emitted by the light sourcewith an energy which permits the light reflected from an object to bedetected despite the effect of sunlight without the injuries which maybe caused by the light from laser sources or focused LEDs impingingdirectly on the eye. The combination of pulsed light sources withstructured illumination and the use of an image sensor array, preferablya CMOS/CCD image sensor, with global shutter unit and variableintegration times in the microsecond and nanosecond range, constitutes aparticularly advantageous system for seat supervision which is e.g.suitable for controlling the activation state of an airbag.

By creating a difference image the present invention thus makes itpossible to eliminate extraneous light effects in the form of directsunlight, shadows, frontal light, street lighting and so on. Only theinvisible, narrow-band near-infrared region of the light sourcecontributes to the signal to be evaluated.

Further developments of the present invention are described in thedependent claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the present invention are described in moredetail below making reference to the enclosed drawings, in which

FIG. 1 shows a schematic representation of a device for seat supervisionaccording to the present invention;

FIG. 2 shows a schematic circuit diagram of an image sensor element ofan image sensor array of the device for seat supervision according tothe present invention;

FIG. 3 shows a schematic representation of the device for seatsupervision according to the present invention in which this device isused in a high-speed camera; and

FIG. 4a) to c) show schematic representations elucidating the principleof optical seat supervision for airbag control.

DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

A device for seat supervision according to the present invention isshown schematically in FIG. 1, a light source being denoted by thereference numeral 2. The light source 2 is preferably a laser diode orLED, which preferably emits light in the invisible near-infrared range,700 nm to 1100 nm. In the embodiment shown an optical unit 4 for forminga fan of light is arranged in front of the light source 2. The unit 4 ispreferably a cylindrical optics unit or a diffraction grating.

The light beam 6 emitted by the light source 2 and the optional opticalunit 4 impinges on an object, which is represented schematically by thecontinuous line 8, and is cast onto an image sensor array 12 by areceive optical unit 10. The image sensor array 12 is preferably aCMOS/CCD image sensor having a resolution of at least 128×128 pixels andwhich has a globally adjustable synchronous illumination time which canbe controlled independently of the readout clock of the pixel values.This provides the possibility of illuminating even in the microsecondand nanosecond range. To achieve this short illumination time, the lightsource 2 is operated using very short pulses of a suitable length, thelight source 2 and the image sensor array 12 being operated insynchronism, this being indicated schematically by the line 14. For thispurpose an electronic control unit 16 is provided for the CMOS/CCD imagesensor and the illumination source for synchronizable illumination. Theelectronic control unit can be realized either as an FPGA (FieldProgrammable Gate Array) or as an ASIC (Application Specific IntegratedCircuit) or, alternatively, it can be integrated together with the imagesensor array on an integrated circuit.

The electronic control unit 16, in conjunction with the image sensorarray, which has a global electronic shutter unit via which theillumination time of the image sensor array can be adjustedindependently of the readout clock of the image sensor array, makes itpossible to record an image with illumination and an image withoutillumination which are separated by less than 10 ms. With the device forseat supervision according to the present invention it is thereforepossible to record a scene with an image repetition rate of 100 to 1000images/s, thus guaranteeing airbag control in the millisecond range. Thetime interval between the two recorded images referred to above, onewith active illumination and one without active illumination, must liebelow 10 ms in order that continuous movements in space do not lead tomisinterpretations of the difference images which form the basis of thesignal evaluation. The difference image can be formed directly on theCMOS/CCD image sensor chip or, alternatively, in a field programmablegate array, an application-specific integrated circuit, a microprocessoror a microcomputer. The unit for forming the difference can thus be partof the electronic control unit or can be implemented independentlythereof.

In preferred embodiments of the device for seat supervision according tothe present invention an interference filter is also provided, thisbeing simply shown schematically in FIG. 1 where it is denoted by thereference numeral 18. The pass frequencies of the optional interferencefilter disk 18 are tuned to the wavelength of the light beam emitted bythe monochromatic light source 2. Through this measure the interferinglight can be suppressed still further. Although the interference filterdisk 18 is located in front of both the image sensor 12 and the lightsource 2 in FIG. 1, it suffices as an option just to place it in frontof the image sensor 12 or the receive optical unit 10. Furthermore, inpreferred embodiments of the device for seat supervision according tothe present invention a signal processing electronic unit is provided,indicated schematically in FIG. 1 by the connection 20, which serves toevaluate the difference image and to connect the signal which suppliesinformation on the occupation of the seat to the airbag control unit.

The signal processing electronic unit uses the difference signalscreated by the image sensor array to perform both object recognition andalso a distance measurement based on the triangulation principle, whichwill now be discussed briefly in terms of FIG. 1. The light source 2projects a light beam 6 at an angle α into a scene in which there ise.g. an object 8.

This results in a light spot at the position x on the image sensor arrayif the object is situated at the distance indicated by the line 8. Ifthe object is closer to the image sensor, it creates a light spot at theposition y on the image sensor array, whereas if it is further away fromthe image sensor it creates a light spot at the position z on the imagesensor array. Since the angles and distances of the light source and theimage sensor in such a device are known, the distance of an object fromthe image sensor can be inferred by simple projection geometry from theposition of the light spot created on the image sensor array 12, e.g. byusing the formula h=d tan (α).

Making reference to FIG. 2, an embodiment for an image sensor element ofthe image sensor array, preferably a CMOS/CCD image sensor, will now bedescribed. Preferably an inexpensive camera system for high-speedcinematography with a sensor chip on the basis of standard CMOStechnology is used as the image sensor. Such a system is cheaper thanclassical optomechanical or optoelectrical high-speed recording systems.In contrast to the fixed structures of CCD image elements, CMOStechnology provides a flexible arrangement of image elements. Apart fromthe cointegration of signal readout and signal processing, manydifferent functions can be realized on the chip. This also makes itpossible to reduce the power loss and the manufacturing costs of thewhole system.

The CMOS sensor for high-speed cinematography is preferably manufacturedin a standard CMOS process with an n well depth of 1 μm with 128×128pixels and has a linear transfer characteristic in respect of theincident light. The CMOS sensor has a global electronic shutter whichpermits simultaneous imaging in all the pixels. This global electronicshutter is needed in order to avoid any possible blurring caused by fastmoving objects. This shutter also enables the illumination time and thereadout clock to be completely divorced from one another.

A sensor element of such a CMOS sensor, which represents a pixel, andemploying a photoelement PD, is shown schematically in FIG. 2. Inpreferred embodiments of the present invention ann⁺-diffusion-p-substrate photodiode is used as the photoelement PDbecause of its fast pulse response to the incident light. The cathode ofthe photodiode is connected to a reference voltage V_(ref) via a resetswitch M1 in the form of a transistor. The anode of the photodiode isgrounded. Light impinging on the photodiode PD generates a photocurrentI_(ph). The cathode of the photodiode PD is also connected via a shuttertransistor M2 to a first terminal of a storage capacitor C_(s), thesecond terminal of which is grounded. The storage capacitor C_(s) isalso connected via its first terminal to the transistors M3 and M4,which are provided to read out the charge stored on the storagecapacitor C_(s).

At the start of recording each and every image, global control signals(global in the sense that they are used for all the sensor elements) 40(reset signal) and 42 (shutter signal) are set to a high level in eachpixel so as to charge the storage capacitor C_(s) up to the referencevoltage V_(ref). The transistors M1 and M2 function here as switches.After the storage capacitor C_(s) has been charged up, the reset signal40 is set to 0, whereupon the integration commences. Depending on thelocal distribution of the photocurrent caused by the externalillumination, the storage capacitor C_(s) is discharged in every pixel.The integration terminates when the shutter signal 42 is set to 0. Thereadout via the transistors M3 and M4 of the charge stored on thestorage capacitor C_(s) can now commence, the row-by-row readout beingactivated by setting a row-select signal 44 to high.

Such a CMOS sensor, and the complete camera system realized therewith,fulfill all the requirements of modern recording systems as regards highreadout rates, i.e. more than 1000 images per second, no blurring, noblooming effects, very short illumination times and a linear transfercharacteristic. These features are of essential importance, particularlyfor recording fast moving objects. This sensor is thus admirablysuitable for seat supervision in motor vehicles by means of which timelycontrol of the activation state of airbags can be achieved.

An image sensor array, which consists of a plurality, e.g. 128×128pixels, of the image sensor elements shown in FIG. 2, constitutes themain component of a CMOS high-speed camera, the control or signalprocessing electronics of which can e.g. be realized in the form of anFPGA module, as can be seen in FIG. 3. In FIG. 3 the sensor module,shown schematically, is connected to the FPGA module 52, which is alsoconnected to an LED/laser diode 54, which preferably operates in theinvisible infrared range, so as to enable synchronous operation of thesensor module 50 and the light source 54. As explained above, the FPGAmodule preferably includes the electronic control unit of the seatsupervision device. The FPGA module is preferably connected via aninterface module 56 to a computation unit 58, which may preferably be amicroprocessor or a microcomputer.

A preferred procedure for performing seat supervision by means of thecomputation unit 58 from the recorded image signals or differencesignals will now be explained in more detail making reference to FIG.4a) to 4 c). This preferred procedure is based on the light sectionprinciple, using which it is possible to determine the contour of anobject on the seat under supervision and thus to identify the object onthe seat. This identification can be performed using methods of patternrecognition, so that e.g. a child's seat can be identified from thesection image.

In addition it is possible from the section image to determine thedistance of an object or a person from the airbag cover using thetriangulation principle, which has been explained above. Using multizonesupervision it can then be decided whether and how the airbag is to beactivated in the event of a collision. Depending on the distance of theperson from the airbag cover, a decision must be made as to whether theairbag is to be activated or deactivated, or whether the power of theairbag should be adjusted to match the distance which has been found orwhether multistage switch-on is needed.

This will now be explained in more detail making reference to FIG. 4a)to 4 c). In FIG. 4a) a person 70 is shown sitting on the seat 72 of amotor vehicle and leaning forwards. The fact that the person is leaningforwards is registered by the seat supervision device according to thepresent invention. The fan of light from the light source 54 isrepresented schematically in each part of FIG. 4. The seat supervisiondevice according to the present invention creates a light section,denoted by the reference numeral 80 in the right-hand part of FIG. 4a).As can be seen from the figure, the recording zone is divided into threesections, an airbag deactivation section 82, a section 84 forpower-adjusted airbag firing, and an airbag activation section 86. Inthe situation represented in FIG. 4a) the seat supervision devicetherefore establishes that the occupant 70 is partly located in theairbag deactivation zone 82 and thus at too short a distance from theairbag 100. As a consequence the airbag is deactivated.

In FIG. 4b) the occupant 70 is shown in a normal sitting position, thusresulting in the section image 90 shown in the right-hand part of FIG.4b). As can be seen, the section image is located completely in theairbag activation region 86, so that the airbag is activated normally soas to fire in the event of a collision. For the section images shown inFIG. 4a) and 4 b) it is established by pattern recognition that in eachcase an occupant 70 is positioned on the seat 72 of the motor vehicle.

In FIG. 4c) the seat 72 of the motor vehicle is unoccupied. This resultsin the section image 92 shown in the right-hand part of FIG. 4c) andthis can be evaluated by pattern recognition as revealing that the seatis unoccupied, so that the airbag 100 can be deactivated in allcircumstances.

Possible positions, 94 and 96, where the seat supervision deviceaccording to the present invention may be located are shown in FIG. 4c).The measurement system of the seat supervision device according to thepresent invention, as has already been explained above, can be mountedat some suitable place in the motor vehicle, e.g. in the rooflining, onthe instrument panel or in the cowl. In FIG. 4c) the instrument panel isdenoted by the reference numeral 94 and the rooflining by the referencenumeral 96.

To persons skilled in the art it is obvious that the device and themethod according to the present invention can be used advantageously toperform airbag activation control in motor vehicles. It is important torealize that the system according to the present invention can beappropriately modified so as to construct a three-dimensional camerasystem which can supervise the complete interior of a motor vehicle,i.e. all the danger zones associated with the numerous airbags in amotor vehicle, e.g. the side airbags, and thus to control appropriatelythe activation state of the respective airbags.

What is claimed is:
 1. A device for seat supervision in motor vehiclescomprising: a pulsed monochromatic light source for the short-timeillumination of a supervised space; an image sensor array for recordingan image of the supervised space which has a global electronic shutterwhich enables the illumination time of the image sensor array to beadjusted independently of a readout clock of the image sensor array; asynchronizer for synchronizing the operation of the pulsed light sourceand the image sensor array; a controller for controlling the lightsource and the image sensor array so as to create a first image of thesupervised space during illumination of the supervised space at a firsttime and to create a second image of the supervised space withoutillumination of the supervised space a second time; and a differenceimage former for creating a difference image from the first and thesecond image.
 2. A device according to claim 1, wherein the illuminationtime of the light source lies in the microsecond or nanosecond range andwherein the first and the second times are at most 10 ms apart.
 3. Adevice according to claim 1, further comprising an interference lightfilter, a pass frequency of which is tuned to a wavelength of theillumination light of the light source, in front of the image sensorarray.
 4. A device according to claim 1, wherein the light source is alaser diode or light emitting diode which emits light in an invisiblenear-infrared range and which is provided with cylindrical optics or adiffraction grating for forming a fan of light.
 5. A device according toclaim 1, wherein an evaluator for evaluating the difference image or aplurality of difference images is also provided so as to control theactivation state of an airbag on the basis of the evaluation.
 6. Adevice according to claim 5, wherein the evaluator evaluates thedifference image or difference images using a triangulation method.
 7. Amethod for seat supervision in motor vehicles, comprising the followingsteps: a) illuminating a supervised space using a pulsed monochromaticlight source employing an illumination time in the microsecond ornanosecond range; b) recording a first image of the supervised spaceduring the illumination in step a) using an image sensor array; c)recording a second image of the supervised space, without illuminatingthe supervised space with the light source, a maximum of 10 ms before orafter the first image; and d) creating a difference image from the firstand the second image.
 8. A method according to claim 7, which alsoincludes the step of controlling an activation state of one or moreairbags on the basis of one or more difference images.
 9. A methodaccording to claim 8, wherein a triangulation method is performed on thebasis of one or more difference images so as to control the activationstate of one or more airbags on the basis of the results of thetriangulation method.